feat: add TPS workload with netperf TCP_RR and HTTP file transfer#38
Merged
Conversation
Workloads like TPS need custom parameters (file-size, iterations, duration) that don't apply to all workloads. This adds a generic Params map[string]string field to WorkloadConfig, merged from YAML config in the run command orchestration. Signed-off-by: Melvin Hillsman <mhillsma@redhat.com>
Add a tps workload that measures transactions per second between server/client VM pairs using two complementary methods: - netperf TCP_RR: transport-layer request/response latency - HTTP file transfer: application-layer TPS via looped GET downloads Key design decisions: - Single systemd unit per role for unified journalctl output - Server readiness wait (curl probe on :8080) prevents wasted iterations - Data port pinned to 12866 for masquerade NAT compatibility - No UDP_RR (unreliable through OpenShift masquerade NAT) - Configurable via Params: file-size, iterations, duration Signed-off-by: Melvin Hillsman <mhillsma@redhat.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mrhillsman The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
exe-prow-github-app
Bot
added
the
approved
Add TPS to the deploying-workloads guide VM table and available workloads list. Signed-off-by: Melvin Hillsman <mhillsma@redhat.com>
Signed-off-by: Melvin Hillsman <mhillsma@redhat.com>
mrhillsman
force-pushed
the
client-server-tps
branch
from
8279bf0 to
aaeb9b6
Compare
Contributor
Author
|
Verified workload deploys and runs: [root@virtwork-tps-client-0 ~]# journalctl -u virtwork-tps.service -b
May 21 03:49:09 virtwork-tps-client-0 systemd[1]: Started virtwork-tps.service - Virtwork TPS client (netperf TCP_RR + HTTP file transfer).
May 21 03:49:09 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: === Virtwork TPS Client ===
May 21 03:49:09 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Server: virtwork-tps-server.virtwork.svc.cluster.local
May 21 03:49:09 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Iterations: 2
May 21 03:49:09 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Duration: 10s per iteration
May 21 03:49:09 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Waiting for server...
May 21 03:49:11 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Server is ready.
May 21 03:49:11 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: ==========================================
May 21 03:49:11 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: TCP_RR TPS Test
May 21 03:49:11 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: ==========================================
May 21 03:49:11 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Msg size: 64 bytes
May 21 03:49:11 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: --- TCP_RR iteration 1/2 ---
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to virtwork-tps-server.virtwork.svc.cluster.loca>
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: Local /Remote
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: Socket Size Request Resp. Elapsed Trans.
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: Send Recv Size Size Time Rate
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: bytes Bytes bytes bytes secs. per sec
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: 16384 131072 64 64 10.00 3356.74
May 21 03:49:21 virtwork-tps-client-0 virtwork-tps-client.sh[1206]: 16384 131072
May 21 03:49:23 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: --- TCP_RR iteration 2/2 ---
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to virtwork-tps-server.virtwork.svc.cluster.loca>
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: Local /Remote
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: Socket Size Request Resp. Elapsed Trans.
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: Send Recv Size Size Time Rate
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: bytes Bytes bytes bytes secs. per sec
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: 16384 131072 64 64 10.00 3569.32
May 21 03:49:33 virtwork-tps-client-0 virtwork-tps-client.sh[1216]: 16384 131072
May 21 03:49:35 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: ==========================================
May 21 03:49:35 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: HTTP File Transfer TPS Test
May 21 03:49:35 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: ==========================================
May 21 03:49:35 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: File size: 10M
May 21 03:49:35 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: --- HTTP iteration 1/2 ---
May 21 03:49:45 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Completed: 258 transfers in 10s
May 21 03:49:45 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Throughput: 25 transfers/sec, 258 MB/s
May 21 03:49:47 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: --- HTTP iteration 2/2 ---
May 21 03:49:57 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Completed: 251 transfers in 10s
May 21 03:49:57 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: Throughput: 25 transfers/sec, 251 MB/s
May 21 03:49:59 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: ==========================================
May 21 03:49:59 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: TPS testing complete
May 21 03:49:59 virtwork-tps-client-0 virtwork-tps-client.sh[1185]: ==========================================
May 21 03:49:59 virtwork-tps-client-0 systemd[1]: virtwork-tps.service: Deactivated successfully.
May 21 03:49:59 virtwork-tps-client-0 systemd[1]: virtwork-tps.service: Consumed 11.916s CPU time, 3.9M memory peak. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Design
The TPS workload deploys server/client VM pairs as a MultiVMWorkload:
Server VM runs a single script (virtwork-tps-server.sh) that:
Client VM runs a single script (virtwork-tps-client.sh) that:
Key decisions:
Configuration
Test plan
Running Workload
❯ ./virtwork run --workloads tps --config ./config.yaml --ssh-user fedora --ssh-password fedora Namespace virtwork ensured Service virtwork-tps-server created Secret virtwork-tps-server-0-cloudinit created Secret virtwork-tps-client-0-cloudinit created VM virtwork-tps-client-0 created VM virtwork-tps-server-0 created Waiting for 2 VMs to become ready (timeout: 10m0s)... VM virtwork-tps-client-0: VMI not yet created, retrying... VM virtwork-tps-server-0: VMI not yet created, retrying... All 2 VMs ready ================================================== Deployment Summary ================================================== Run ID: 72542978-703f-4a58-9394-5d0e4cec83e2 Namespace: virtwork VMs created: 2 Services: 1 Secrets: 2 Image: quay.io/containerdisks/fedora:41 ==================================================Justification
We have a partner running these tests but we do not have visibility into what they are running exactly only that they are doing client/server tps with a minimum of client performing HTTP GET file from server. We emulate that interaction and add a small 64 byte TCP packet transaction.